What do a 20th-century physicist, an 18th-century statistician and an ancient Greek philosopher have in common? They all knew how to extrapolate with incredible accuracy. Suppose I showed you a box and asked you to guess what is inside, without providing any more details. You might think this is completely impossible, but the nature of the container provides some information - the contents must be smaller than the box, for example, while a solid metal box can hold liquids and withstand temperatures that a cardboard box would struggle with. Is there a way to describe this process of guessing with limited information in a mathematically sensible way?

A core tension in models of concept learning is that the model must carefully balance the tractability of inference against the expressivity of the hypothesis class. Humans, however, can efficiently learn a broad range of concepts. We introduce a model of inductive learning that seeks to be human-like in that sense.It implements a Bayesian reasoning process where a language model first proposes candidate hypotheses expressed in natural language, which are then re-weighed by a prior and a likelihood.By estimating the prior from human data, we can predict human judgments on learning problems involving numbers and sets, spanning concepts that are generative, discriminative, propositional, and higher-order.

Learning to reason and carefully explain arguments is central to students' cognitive, mathematical, and computational thinking development. This is particularly challenging in problems under uncertainty and in Bayesian reasoning. With the new generation of large language models (LLMs) capable of reasoning using Chain-of-Thought (CoT), there is an excellent opportunity to learn with them as they explain their reasoning through a dialogue with their artificial internal voice. It is an engaging and excellent opportunity to learn Bayesian reasoning. Furthermore, given that different LLMs sometimes arrive at opposite solutions, CoT generates opportunities for deep learning by detailed comparisons of reasonings. However, unlike humans, we found that they do not autonomously explain using ecologically valid strategies like natural frequencies, whole objects, and embodied heuristics. This is unfortunate, as these strategies help humans avoid critical mistakes and have proven pedagogical value in Bayesian reasoning. In order to overcome these biases and aid understanding and learning, we included prompts that induce LLMs to use these strategies. We found that LLMs with CoT incorporate them but not consistently. They show persistent biases towards symbolic reasoning and avoidance or phobia of ecologically valid strategies.

A core tension in models of concept learning is that the model must carefully balance the tractability of inference against the expressivity of the hypothesis class. Humans, however, can efficiently learn a broad range of concepts. We introduce a model of inductive learning that seeks to be human-like in that sense.It implements a Bayesian reasoning process where a language model first proposes candidate hypotheses expressed in natural language, which are then re-weighed by a prior and a likelihood.By estimating the prior from human data, we can predict human judgments on learning problems involving numbers and sets, spanning concepts that are generative, discriminative, propositional, and higher-order.

The symbolic AI community is increasingly trying to embrace machine learning in neuro-symbolic architectures, yet is still struggling due to cultural barriers. To break the barrier, this rather opinionated personal memo attempts to explain and rectify the conventions in Statistics, Machine Learning, and Deep Learning from the viewpoint of outsiders. It provides a step-by-step protocol for designing a machine learning system that satisfies a minimum theoretical guarantee necessary for being taken seriously by the symbolic AI community, i.e., it discusses "in what condition we can stop worrying and accept statistical machine learning." Unlike most textbooks which are written for students trying to specialize in Stat/ML/DL and willing to accept jargons, this memo is written for experienced symbolic researchers that hear a lot of buzz but are still uncertain and skeptical. Information on Stat/ML/DL is currently too scattered or too noisy to invest in. This memo prioritizes compactness, citations to old papers (many in early 20th century), and concepts that resonate well with symbolic paradigms in order to offer time savings. It prioritizes general mathematical modeling and does not discuss any specific function approximator, such as neural networks (NNs), SVMs, decision trees, etc. Finally, it is open to corrections. Consider this memo as something similar to a blog post taking the form of a paper on Arxiv.

Memristive devices that mimic neuron-connecting synapses could serve as the hardware for neural networks that copy the way the brain learns. Now two new studies may help solve key problems these components face not just with yields and reliability, but with finding applications beyond neural nets. Memristors, or memory resistors, are essentially switches that can remember which electric state they were toggled to after their power is turned off. Scientists worldwide aim to use memristors and similar components to build electronics that, like neurons, can both compute and store data. These memristive devices may greatly reduce the energy and time lost in conventional microchips shuttling data back and forth between processors and memory.

A few years ago, at the Princeton Club in Manhattan, I chanced on a memorable chat with the Harvard psychologist Steven Pinker. His spouse, the philosopher Rebecca Goldstein, with whom he was tagging along, had been invited onto a panel to discuss the conflict between religion and science and Einstein's so-called "God letter," which was being auctioned at Christie's. Pinker had recently published Enlightenment Now: The Case for Reason, Science, Humanism, and Progress. I was eager to pepper him with questions, mainly on religion, rationality, and evolutionary psychology. I remember I wanted Pinker's take on something Harvey Whitehouse, one of the founders of the cognitive science of religion, told me in an interview--that my own little enlightenment, of becoming an atheist in college, was probably mostly a product of merely changing my social milieu. I wasn't so much moved by rational arguments against the ethics and existence of God but by being distanced from my old life and meeting new, non-religious friends. I recall Pinker almost pouncing on that argument, defending reason's power to change our minds. He noted that people especially high in "intellectance," a personality trait now more commonly called "openness to experience," tend to be more curious, intelligent, and willing to entertain new ideas. I still think that Pinker's way of seeing things made more sense of my experience in those heady days. I really was, for the first time, trying my best to think things through, and it was exhilarating. We talked until the event staff shelved the wine, and parted ways at a chilly midtown intersection.

We access the internalized understanding of trained, deep neural networks to perform Bayesian reasoning on complex tasks. Independently trained networks are arranged to jointly answer questions outside their original scope, which are formulated in terms of a Bayesian inference problem. We solve this approximately with variational inference, which provides uncertainty on the outcomes. We demonstrate how following tasks can be approached this way: Combining independently trained networks to sample from a conditional generator, solving riddles involving multiple constraints simultaneously, and combine deep-learned knowledge with conventional noisy measurements in the context of high-resolution images of human faces.

The coefficients are constrained by the prior and end up smaller in the second example. Although the model is not fit here with Bayesian techniques, it has a Bayesian interpretation. The output here does not quite give a distribution over the coefficient (though other packages can), but does give something related: a 95% confidence interval around the coefficient, in addition to its point estimate. By now you may have a taste for Bayesian techniques and what they can do for you, from a few simple examples. Things get more interesting, however, when we see what priors and posteriors can do for a real-world use case. For part 2, please click here.

From within the dark confines of the skull, the brain builds its own version of reality. By weaving together expectations and information gleaned from the senses, the brain creates a story about the outside world. For most of us, the brain is a skilled storyteller, but to spin a sensible yarn, it has to fill in some details itself. "The brain is a guessing machine, trying at each moment of time to guess what is out there," says computational neuroscientist Peggy Seriès. Guesses just slightly off -- like mistaking a smile for a smirk -- rarely cause harm.